Search for: All records

In this paper, we focus on the important yet understudied problem of Continual Federated Learning (CFL), where a server communicates with a set of clients to incrementally learn new concepts over time without sharing or storing any data. The complexity of this problem is compounded by challenges from both the Continual and Federated Learning perspectives. Specifically, models trained in a CFL setup suffer from catastrophic forgetting which is exacerbated by data heterogeneity across clients. Existing attempts at this problem tend to impose large overheads on clients and communication channels or require access to stored data which renders them unsuitable for real-world use due to privacy. We study this problem in the context of Foundation Models and showcase their effectiveness in mitigating forgetting while minimizing overhead costs and without requiring access to any stored data. We achieve this by leveraging a prompting based approach (such that only prompts and classifier heads have to be communicated) and proposing a novel and lightweight generation and distillation scheme to aggregate client models at the server. We formulate this problem for image classification and establish strong baselines for comparison, conduct experiments on CIFAR-100 as well as challenging, large-scale datasets like ImageNet-R and DomainNet. Our approach outperforms both existing methods and our own baselines by more than 7% while significantly reducing communication and client-level computation costs. 

The influence of additives on the detonation velocity of a polyethylene wax/RDX formulation was examined. Additives included species of various shock impedance: glass microballoons; glass microspheres; polymethyl methacrylate (PMMA) microspheres; thermally expandable microspheres (TEMs); and PMMA microencapsulated pentaerythritol tetranitrate (PETN). Performance of the insensitive explosive 2,4-dinitroanisole (DNAN) was enhanced by addition of PETN-either neat or encapsulated, but encapsulation did not increase the sensitivity of the formulation. The energy contribution of the encapsulated PETN to the detonation front of the insensitive explosive 2,4-dinitroanisole (DNAN) was also demonstrated. Present in 5 wt%, the encapsulated PETN allowed DNAN to sustain a reaction (5.36 km/s) at 13 mm, well below its critical diameter. 

Continual learning is a setting where machine learning models learn novel concepts from continuously shifting training data, while simultaneously avoiding degradation of knowledge on previously seen classes which may disappear from the training data for extended periods of time (a phenomenon known as the catastrophic forgetting problem). Current approaches for continual learning of a single expanding task (aka class-incremental continual learning) require extensive rehearsal of previously seen data to avoid this degradation of knowledge. Unfortunately, rehearsal comes at a cost to memory, and it may also violate data-privacy. Instead, we explore combining knowledge distillation and parameter regularization in new ways to achieve strong continual learning performance without rehearsal. Specifically, we take a deep dive into common continual learning techniques: prediction distillation, feature distillation, L2 parameter regularization, and EWC parameter regularization. We first disprove the common assumption that parameter regularization techniques fail for rehearsal-free continual learning of a single, expanding task. Next, we explore how to leverage knowledge from a pre-trained model in rehearsal-free continual learning and find that vanilla L2 parameter regularization outperforms EWC parameter regularization and feature distillation. Finally, we explore the recently popular ImageNet-R benchmark, and show that L2 parameter regularization implemented in self-attention blocks of a ViT transformer outperforms recent popular prompting for continual learning methods. 

Seasonal epidemics and sporadic pandemics of influenza cause a large public health burden. Although influenza viruses disseminate through the environment in respiratory secretions expelled from infected individuals, they can also be transmitted by contaminated surfaces where virus-laden expulsions can be deposited.

 

Computer vision models suffer from a phenomenon known as catastrophic forgetting when learning novel concepts from continuously shifting training data. Typical solutions for this continual learning problem require extensive rehearsal of previously seen data, which increases memory costs and may violate data privacy. Recently, the emergence of large-scale pre-trained vision transformer models has enabled prompting approaches as an alternative to data-rehearsal. These approaches rely on a key-query mechanism to generate prompts and have been found to be highly resistant to catastrophic forgetting in the well-established rehearsal-free continual learning setting. However, the key mechanism of these methods is not trained end-to-end with the task sequence. Our experiments show that this leads to a reduction in their plasticity, hence sacrificing new task accuracy, and inability to benefit from expanded parameter capacity. We instead propose to learn a set of prompt components which are assembled with input-conditioned weights to produce input-conditioned prompts, resulting in a novel attention-based end-to-end key-query scheme. Our experiments show that we outperform the current SOTA method DualPrompt on established benchmarks by as much as 4.5% in average final accuracy. We also outperform the state of art by as much as 4.4% accuracy on a continual learning benchmark which contains both class-incremental and domain-incremental task shifts, corresponding to many practical settings. 

In introductory biology classrooms, cell and molecular concepts are often taught separate from those related to evolution and ecology, and usually in completely different courses. Furthermore, many examples used to teach introductory concepts are difficult for students to relate to. To address these issues, we developed curricular materials focused on the topic of breast cancer that: (1) aim to teach students how to integrate the various sub-disciplines of biology, with evolution as the unifying theme, and (2) aim to present course materials using relatable examples such as human health and disease. To assess the potential value of these materials, we asked students to complete a pre-unit and post-unit assessment before and after completing the interactive course unit on breast cancer.

We found that after learning about breast cancer, students reported that learning about biology in the context of human health made their learning experience easier, more interesting, and more relatable. After the unit, students also rated evolutionary concepts as being more important for understanding human health and disease.

These results have important implications for developing introductory biology curricula that have more personal appeal to students and may thus translate to better learning outcomes, as well as help students better understand the process of evolution as it occurs in humans.

 

The commercial production and subsequent movement of bumble bees for pollination of agricultural field and greenhouse crops is a growing industry in North America and globally. Concerns have been raised about the impacts of pathogen spillover from managed bees to wild pollinators, including from commercial bumble bees. We recommend development of a program to mitigate disease risk in commercial bumble bee production, which will in turn reduce disease stressors on wild pollinators and other insects. We provide recommendations for the components of a clean stock program with specific best management practices for rearing commercial bumble bees including related products such as wax, pollen, and nesting material.